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(54) Subtitle recording and reproduction 

(57) Subtitles of e.g. video information are recorded 
on a record medium (91) e.g. a disc recording. A first 
subtitle is recorded for use during normal playback. A 
second subtitle is recorded for use exclusively during 
trick-play. Addresses for the subtitles to be played back 
during the trick playback mode are stored in an area of 
the record medium (91) reserved for these addresses. 
The record medium acts as a computer-readable mem- 
ory which directs a computer (14) to retrieve these ad- 
dresses and locate subtitles to be displayed (1 ,7) during 
the trick playback mode on the record medium. The sub- 
titles are searched by decoding (7) the subtitles located 
at the addresses on the record medium The decoded 
(7) subtitles are displayed during the trick playback 
mode, mixed with the video (8,9,10). 
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Description 

The present invention relates to: recording and decoding subtitles on, and reproducing subtitles from, a record 
medium; and to a computer-readable memory useful in reproducing a subtitle. 

5 Television broadcasting or video reproduction (such as from a video disk) provides subtitles superimposed on the 

video image. Problematically, the subtitles are permanently combined with the underlying video image and cannot be 
manipulated at the receiving (or reproducing) end. The subtitles, for example, cannot be searched for information 
concerning a specific scene occurring in the video image or sound in its corresponding audio track. 

Compact Disc Graphics (CD-G) provide some flexibility in searching subtitles because this technique records 

10 graphics in the form of subcodes. However, CD-G has a serious disadvantage because this technique is limited to 
compact disc (CD) applications, which are slow by television standards. That is, the CD-G technique does not lend 
itself to manipulation of subtitles in real-time television broadcasts or video reproductions. 

As will be shown with reference to Figs. 18A-C and 19, the lead time required to generate a full CD-G screen is 
grossly inadequate for normal television or video broadcasts. Fig. 18A depicts the CD-G data format in which one 

15 frame includes 1 byte of a subcode and 32 bytes of audio channel data, 24 bytes of which are allocated for L and R 
audio channel data (each channel having 6 samples with 2 bytes per sample) and 8 bytes allocated toan error correction 
code The frames are grouped as a block of 98 frames (Frame 0, Frame 1, .... Frame 96 and Frame 97) as shown in 
Fig. 18B and eight of these blocks RQ,R,S,T,U,V and W are transmitted as shown in Fig 18C. The subcodes for 
Frames 0 and 1 in each block are reserved for sync patterns SO, S1 , whereas the subcodes for the remaining 96 frames 

20 are reserved for various subcode data. The first two blocks R Q are allocated to search data employed for searching 
through record tracks, while the remaining 6 blocks R,S,T,U,V and W are available for graphic data. 

CD-G transmits each block of 98 frames at a repeating frequency of 75 Hz. Thus, the data transmission rate for 
1 block is (75 Hz x 98 bytes) 7.35 kHz, resulting in a subcode bit rate of 7.35 K bytes/s. 

The transmission format for transmitting the information present in blocks R,S,T,U,V and W is shown in Fig. 19, 

25 wherein each of the 96 frames (2,3,... 97) of the 6 blocks (R,S,T,U,V and W) is transmitted as a packet including 6 
channels (R to W) of 96 symbols per channel. The packet is further subdivided into 4 packs of 24 symbols apiece 
(symbol 0 to symbol 23), with each pack storing a CD-G character It will be appreciated that, a CD-G character is 
made up of 6 x 12 pixels and, therefore, is easily accommodated in each 6 x 24 pack. According to the CD-G format, 
the 6 x 12 CD-G character is stored in the six channels of (R,S,T,U,V and W) at symbols 8 to 19 (12 symbols). The 

30 remainder of the symbols in each of the packs store information about the character. 

Mode information is one example of information stored in the packs and is stored in the first 3 channels (R, S : T) 
of symbol 0 in each pack. Item information is another example which is stored in the last 3 channels (U, V W) of symbol 
0. A combination of the mode information and the item information defines the mode for the characters stored in the 
corresponding pack as follows: 

35 



Table 1 



40 



Mode 


Item 




000 


000 


0 mode 


001 


000 


graphics mode 


001 


001 


TV-graphics mode 


111 


000 


user's mode 



An instruction is another example of information stored in the packs and is stored in all of the channels of symbol 
45 1 . Corresponding mode, item, parity or additional information for the instruction is stored in ail of the channels of symbols 

2 to 7. Parity information for all of the data in the channels of symbols 0 to 1 9 is stored in all of the channels of the last 

4 symbols (symbols 20 to 23) of each pack. 

As discussed, the CD-G system is slow. The CD-G data is transmitted at a repeating frequency of 75 Hz and, 

therefore, a packet which contains 4 packs is transmitted at a rate of 300 packs per second (75 Hz x 4 packs). That 
50 is, with 1 character allocated to the range of 6 x 12 pixels, 300 characters can be transmitted in 1 second. However, 

a CD-G screen is defined as 288 horizontal picture elements x 192 CD-G vertical picture elements and requires more 

than twice the 300 characters transmitted in 1 second The total transmission time for a 288 x 192 screen is 2.56 

seconds as shown by the following equation: 

55 

(288/6) x (192/12) + 300 = 2.56 seconds 
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With the CD-G system, searching for a specific event (such as a scene) would be extremely time consuming 
because the time to regenerate each screen (2.56 seconds) by itself is extremely long, when it is considered that 
screens are usually refreshed in tenths of a second. This problem is compounded when hexadecimal codes are used 
for the characters because each hexadecimal expression requires 4 bits to represent 1 pixel. As a result, 4 times the 
$ data described above is transmitted, thereby increasing the transmission rate to 10.24 seconds (4 x 2.56 seconds). 
Since each screen requires a sluggish 10.24 seconds for transmission, a continual transmission of screens means 
that a lag time of 1 0.24 seconds is experienced when transmitting screens using the CD-G technique 

In one type of system (known as the CAPTAIN system), dot patterns, as well as character codes, represent the 
subtitles. This system, however, does not appear to be significantly better than the CD-G system and suffers from 
io some of the same disadvantages. That is, both systems lack the capability to search for a specific event efficiently In 
addition, these systems do not provide subtitles with sufficient resolution power in displaying the subtitles. The CD-G 
system designates only 1 bit for each pixel, and this binary pixel data creates undesired aliasing and flicker. The CAP- 
TAIN system, for example, is developed for a 248 (horizontal picture elements) by 192 (vertical picture elements) 
display, i.e., a low resolution display, and not for high resolution video pictures of 720 x 480. 
15 Aspects of the invention are specified in the Claims. 

An embodiment of the invention seeks to provide an encoding method and apparatus for encoding subtitles to be 
played back exclusively during the trick playback mode, i.e., during fast, slow or reverse playback modes. 

Another embodiment of the invention seeks to provide a computer-readable memory for directing a computer to 
search the subtitles stored in the memory. 
20 A further embodiment of the invention seeks to provide a decoding method and apparatus for decoding the subtitles 

to be displayed during the trick playback mode. 

An embodiment of the present invention provides an encoding method and apparatus which encodes a first subtitle 
to be displayed during the normal playback mode and writes a corresponding first address to the record medium in an 
area reserved for such addresses A second subtitle to be displayed in a respective video frame in place of the first 
25 subtitle is encoded onto the record medium at a corresponding address in the area reserved for such addresses. 

Another embodiment of the present invention further provides a computer-readable memory for directing a com- 
puter to playback a subtitle to be displayed during a trick playback mode by pointing the computer to addresses of 
subtitles to be displayed in the trick playback mode. 

A further embodiment of the present invention provides a decoding method and apparatus which decodes the 
30 subtitles to be displayed exclusively during the trick playback mode by reading out the subtitle from a subtitle address 
stored on the record medium. 

A more complete appreciation of the present invention and many of its attendant advantages will be readily obtained 
by reference to the following illustrative description considered in connection with the accompanying drawings, in which: 

35 Fig. 1 is a block diagram of an illustrative data decoding apparatus of the present invention; 

Fig. 2 is a block diagram of the subtitle decoder depicted in Fig. 1 ; 

Figs. 3A and 3B are tables of addresses used in illustrative apparatus of the present invention; 
Figs. 4A and 4B is a diagram depicting subtitle search operation in normal and trick playback modes; 
Fig. 5 is a table of communications between the system controller of Fig. 1 and the controller of Fig. 2; 
40 Fig. 6 is a table of parameters for the communications between components of Fig. 1 and Fig. 2; 

Figs. 7A to 7C are illustrative signal diagrams demonstrating data encoding used in embodiments of the present 
invention; 

Fig. 8 is a color look up table referred to when encoding subtitle data; 

Fig. 9 is a block diagram of an illustrative encoding apparatus of the present invention; 
45 Figs 1 0A and 1 0B depict a block diagram for the wipe data sampler of Fig. 9; 

Fig 11 is a color look up table referred to when conducting a color wipe operation; 

Fig. 12 is a graph for the explanation of a code buffer operation; 

Fig. 13 is a block diagram describing the internal operation of the code buffer in Fig. 2; 

Figs. 14A to 14C depict a scheme for the colorwiping operation; 
50 Fig. 15 is a block diagram depicting the colorwiping operation according to Figs 14A to 14C; 

Figs. 16A to 16C depict a scheme for the dynamic positioning operation; 

Fig. 17 is a block diagram depicting a circuit for the dynamic positioning operation according to Figs. 16A to 16C; 
Figs. 18A to 18C depict the arrangement of data according to a CD-G format; and 
Fig. 19 depicts a transmission format of the data in the CD-G format. 
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Decoding Apparatus 

The illustrative data decoding apparatus shown in Fig. 1 incorporates the present invention and operates to decode 
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a reproduced signal. A system controller 14 of the data decoding apparatus causes the signal reproduced from, for 
example, a disk 91, to be processed and sent to a subtitle decoder 7. The system controller communicates with a 
controller 35 (Fig. 2) ot the subtitle decoder to decode the subtitles and superimpose them onto a decoded video image 
for display on a television screen. 

5 A data decoder and demultiplexer 1 receives a digital signal reproduced from, for example, a disk, a VCR, or the 

like. The data decoder and demultiplexer error decodes the reproduced signal, preferably employing an Error Correcting 
Code (ECC) technique, and demultiplexes the error decoded reproduced signal into video, subtitle and audio data 
streams. A memory 2 may be used, for example, as a buffer memory as a work area for the purpose of error decoding 
and demultiplexing the reproduced signal. 

10 A video decoder 3 decodes the demultiplexed video data from a video data stream. A memory 4 may be employed 

for the operation of decoding the video data similar to the operation of the memory 2 employed with data decoder and 
demultiplexer 1 . 

A letter box circuit 5 converts the decoded video data with a 4:3 aspect ratio to a 16:9 aspect ratio. The conversion 
is performed using a 4 to 3 decimation process, whereby every four horizontal lines are decimated to three horizontal 
15 lines, thus squeezing the video picture into a 34 picture. According to the letter box format, a vertical resolution com- 
ponent is derived from the remaining Va of the video picture and is employed to enhance the vertical resolution of the 
decimated video picture A timing adjustment memory 6 times the transmission of the video picture to ensure that the 
% of the letter box picture is not transmitted When the decoded video data generated by the video decoder 3 is already 
in a 16:9 letter box format, the letter box circuit bypasses the decimation operation and sends the decoded video data 
20 directly to subtitle decoder 7. 

The decoded subtitle data demultiplexed by the data decoder and demultiplexer 1 is sent directly to subtitle decoder 
7 which decodes the subtitle data according to instructions from system controller 14 and mixes the decoded subtitle 
data with the decoded video data. 

A composite encoder 8 encodes the mixed subtitle data and video data into a suitable video picture format, such 
25 as NTSC, PAL or the like. A mode display 9 interfaces with a user and indicates, for example, the mode of a television 
monitor connected to the illustrated apparatus. A D/A converter 10 converts the encoded signal received from the 
composite encoder into an analog signal suitable for display in the indicated mode, such as NTSC or PAL. 

The audio portion of the audio/video signal decoded by the data decoder and demultiplexer 1 is decoded by an 
audio decoder 11 which decodes the demultiplexed audio data using a memory 12, for example. The decoded audio 
30 data output from the audio decoder is converted into an analog audio signal appropriate for reproduction by a television 
monitor by a D/A converter 1 3. 

Subtitle Decoder 

35 Subtitle decoder 7, as will be discussed with reference to Fig. 2, decodes the encoded subtitle data and mixes the 

decoded subtitle data with the appropriate video data. Controller 35 (Fig. 2) controls the operations of the subtitle 
decoder and communicates with the system controller 14 of the decoder (Fig. 1) using the command signals shown 
in Fig. 2 (as listed in Fig. 5). Together, the controller 35 and system controller 14 time the decoding of the subtitle data 
so that the subtitle data is mixed with video image data at the proper position whereat the subtitles are to appear on 

40 the video image. 

A word detector 20 of the subtitle decoder receives the subtitle data in groups of bit streams reproduced from a 
disk, the bit streams being stored on the disk in packets. Each group of bit streams makes up one frame (or page) of 
subtitles to be superimposed on a video image. Different groups of bit streams may represent subtitles displayed in 
different playback modes, such as normal playback, fast-reverse or fast -forward, alternatively referred to as trick modes. 

45 The system controller indicates to the word detector using a stream_select signal which playback mode is to be 
adopted for display and the word detector selects the appropriate bit stream of signals for the indicated playback mode 
In the case where different video images are displayed on different channels, the system controller indicates the ap- 
propriate channel to the word detector correspondingly in a ch select signal and the word detector changes channels 
to receive only those bit streams on the selected channel. 

50 A group of bit streams making up one frame and received by the word detector includes page header information 

(s.header) which describes the format of the group of bit streams. The page header information is accompanied by 
header error information (header error) and data error information (data error). The system controller uses the page 
header information to determine how to parse the group of bit streams and extract the relevant subtitle data therefrom. 
The system controller uses the header error information to correct anomalies in the page header information and uses 

55 the data error information to correct anomalies in the subtitle data. 

The word detector forwards the subtitle data (Bitmap) along with other decoded information (including a presen- 
tation time stamp PTS, position data position_data and color look up table data CLUT_data) to a code buffer 22. The 
PTS is a signal that indicates the precise time when the audio, video and subtitle data for a frame is transmitted so 
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that the system controller knows when to demultiplex the data from the reproduced signal. The position data indicates 
the horizontal and vertical position where the subtitles are to be superimposed on the video image. The CLUT_data 
indicates which colors are to be used tor the pixels making up the subtitles. For example, the system controller 14 
determines that a video image is being displayed and sends the subtitle data to subtitle decoder 7 at the time indicated 

5 by the time stamp (PTS) and causes the subtitle decoder to output the corresponding subtitle data (Bitmap) at a 
position in the video image represented by the horizontal and vertical position indicated by the position_data in the 
color indicated by the CLUT_data 

The word detector also detects display time data and forwards this data to the system controller (Fig. 1) via 
multiplier 350 of controller 35. (Fig. 2). This display time data is read from the page header information of a group of 

10 bit streams comprising one page (i.e., a frame) of subtitle data. The display time data indicates a time duration in 
which the subtitle encoded in the page is to be displayed with a respective video frame during a normal playback mode. 
When, for example, the system controller (Fig. 1) indicates with the special signal to controller 35 that the system is 
in normal playback mode, the controller causes the multiplier to be bypassed and the display time data is forwarded 
directly to the system controller. In the case where the special signal indicates a special (trick playback) mode, the 

is controller causes the display time data to be switched to the multiplier and the display time data is factored before 
being forwarded to the system controller. It will be appreciated that the display times stored with the trick mode subtitles 
are for the normal playback mode because it is unknown in advance which playback speed the viewer will select and 
that these display times are modified by the controller for different playback speeds. 

By factoring the display time data, the time duration for display of the subtitle in a trick playback mode is varied. 

20 This is particularly advantageous when the display time for a subtitle is, for example, longer than the time duration of 
display for a video frame in trick playback mode. In that situation, displaying the subtitle for a duration equal to the 
unmodified display time would undesirably result in the subtitle being displayed in the trick playback mode for a longer 
duration than the respective video frame is displayed. In other situations, it may be desirable to extend the duration of 
subtitle display beyond a single video frame when, for example, a subtitle to be displayed exclusively in the trick play- 

25 back mode relays information to a viewer about a plurality of the video frames. Accordingly, the multiplier factors the 
display time data to display the subtitle for a specified duration determined by controller 35 depending upon the 
situation. 

A scheduler 21 is provided to ensure that the data received by the code buffer 22 from the demultiplexer 1 (Fig. 
1) does not overflow the code buffer. The scheduler controls read/write access to the code buffer by determining the 

30 bandwidth for an I/O port (not shown) which receives the bit streams selected by the word detector. The bandwidth 
refers to the read/write rate and is calculated by dividing the rate at which the demultiplexer demultiplexes data by the 
number of parallel bits written or read from the code buffer. For example, a data rate from the demultiplexer of 20 Mbps 
divided by 8 parallel bits results in a 2.5 Mbps rate of data read from the code buffer. Therefore, the scheduler will set 
the read/write rate of the I/O port in order to maintain a consistent flow rate of data into and out of the code buffer. The 

35 code buffer, thus, receives the subtitle data (Bitmap) and awaits a decode start signal from the system controller to 
read out the data. 

Advantageously, the system controller executes reading in real time when it is determined from the horizontal and 
vertical sync signals that the television display is at a position corresponding to the position indicated by the position 
data. For real time display, the reading rate should correspond to a picture element sampling rate, preferably 13.5 

^o MHz. As discussed, the subtitle data preferably is written into the code buffer at a rate of 2.5 MHz or more. Thus, the 
13.5 MHz sampling clock is divided into four clock cycles of 3.375 MHz each. One of these 3.375 MHz clock cycles is 
allocated to writing (because writing requires at least 2.5 MHz) and the remaining three clock cycles are allocated to 
reading data from the code buffer, thus satisfying the requirement for real time display. 

The read/write operation described is not only advantageously performed in real time, but also provides high res- 

45 olution Eight bits of the subtitle data are read from the code buffer 22 for each of three read clock cycles, or twenty- 
four bits per sampling clock When display of the picture is conducted by the television monitor every fourth clock cycle, 
one-fourth of the twenty-four bits, (24/4 =) 6 bits are displayed at every clock cycle. That is, each subtitle picture element 
may comprise six bits, which is more than sufficient to achieve a high quality of resolution for the subtitles. 

The operation of the code buffer 22 and corresponding components of Fig. 2 is depicted in the block diagram in 

50 Fig. 1 3. The code buffer 22-1 accumulates bit streams of subtitle data until at least one page of subtitle data is accu- 
mulated in the code buffer. The subtitle data for one page is transferred from the code buffer 22-1 to the display memory 
22-2 (which acts as a buffer for the subtitle decoder) when the subtitle portion of the display time stamp (PTS) is aligned 
with the synchronizing clock (SCR). The synchronizing clock advances a pointer in the display memory 22-2 during 
reading indicating which address of the stored subtitle data is being currently read. It will be noted that placing the code 

55 buffer and display memory in a single unit is preferred since the code buffer need only increment one pointer for pointing 
to the current address in the display memory 22-2 which stores the next set of subtitle data. With an internal memory, 
therefore, virtually no delay is attributed to a transfer operation, resulting in a high speed transfer of the subtitle data. 
When the code buffer is read during a normal playback mode, the synchronizing clock advances the pointer of the 
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display memory 22-2 at each clock pulse. However, during special (or trick) reproduction (such as fast-forward, fast- 
reverse playback modes), the pointer is advanced at a different rate. To this end, a special command is first sent to 
the controller 35 and the controller sends back an acknowledge signal (spec ia Lack), acknowledging that special 
reproduction is to be initiated. To uniformly speed up (or slow down) the operations of the subtitle decoder according 
5 to the special reproduction rate, the system clock reference (SCR) can be altered by adding or subtracting clock pulses. 
Subtraction pulses are created at an n times rate corresponding to the rate of fast-feeding or fast-reverse feeding. For 
example, at the time when special reproduction is commenced, real time subtraction is performed on the bit stream of 
subtitle data read out from Ihe code buffer at the n times rate and the pointer advances at the desired rate to effect the 
special playback mode. 

10 When the special reproduction operation corresponds to a pause operation, on the other hand, no subtraction 

pulses are created. Instead, an identical frame is continuously read from the code buffer repeatedly, thus providing the 
illusion sensation that the subtitles are paused. 

The reading operation is ended when subtitle decoder 7 determines that an end of page (EOP) of the subtitle 
frame is reached. The system controller sends a repeat time signal to the controller 35 which indicates the length of 

15 a page. An inverse run-length circuit 24 includes a counter and sends a display end signal to the controller 35 when 
the count value of the counter reaches the value indicated by the repeat time signal. When controller 35 determines 
that the repeat time is reached, the reading operation of the code buffer is stopped For purposes of this invention, the 
code buffer preferably stores at least two pages of subtitle data because one page will be read while another page is 
written into the code buffer. 

20 Controller 35 issues a buffer overflow signal to system controller 14 when an overflow of code buffer 22 occurs. 

An overflow can be determined when the controller receives the display end signal from inverse run-length circuit 24 
before word detector 20 receives an end of page (EOP) signal on the following page. At that time, the system controller 
withholds transfer of subtitle data from data decoder and demultiplexer 1 (Fig. 1) to the word detector to prevent an 
overflow of the code buffer. When an overflow condition has passed, the next stream will be written into the code buffer 

25 and displayed at the correct display start position. 

An underflow condition exists when code buffer 22 has completed reading the subtitle data for an entire page and 
no further data exists in the code buffer. The code buffer is depicted with a capacity of two pages by the "code buffer 
size" line in Fig. 12. Graphically, an underflow would appear in Fig. 12 as one of the vertical portions of line (C) which 
extends below the lower limit of the code buffer. By contrast, an overflow condition is graphically depicted in Fig. 12 

30 when the subtitle data read into the code buffer is too large, i.e., the horizontal portion of line (C) extends beyond line (B). 

Fig. 12 graphically demonstrates the data flow into and out of code buffer 22. The T-axis (abscissa) represents 
time, while the D-axis (ordinate) represents data size for each page of data. Thus, the gradient (rise/run) represents 
the data flow rate of the subtitles into the code buffer. Graph (C) represents the data flow of the subtitle data. The 
vertical portions of graph (C) indicate a transfer of subtitle data from the code buffer when the display time stamp (PTS) 

35 is aligned with the synchronizing clock (SCR) generated internally by subtitle decoder 7. The horizontal portions of the 
graph (C) indicate the transfer of subtitle data into the code buffer, the display time for normal playback mode indicated 
by the horizontal lines with arrows. For example, at a time that the display time stamp (PTS) for page (SO) is received 
by the code buffer, the previous page of subtitle data is transferred from the code buffer and page (SO) is written into 
the code buffer. When another display time stamp (PTS) is received by the code buffer, the subtitle data of page (SO) 

40 is transferred out of the code buffer and page (S1) is written in. Similarly, the remaining pages (S2), (S3) are written 
into and read out of the code buffer as indicated. 

To precisely time the reading of the subtitle data from the code buffer with the display of the video image, delay 
compensation must be performed to allow for delays within the subtitle decoder. This is especially important where an 
external memory is employed as the display memory because an external memory increases the delay lactor. Delay 

45 compensation is achieved by controlling the timing of the decode start command from system controller 14. The 
system controller delays the decode start command by a time equal to the processing of a letter box picture (approx- 
imately one field) and a delay caused by video decoding at the instant the synchronizing clock of the controller (SCR) 
is aligned with the display time stamp (PTS). Delay compensation is particularly useful, since the video, audio and 
subtitle data are multiplexed on the premise that the decode delay in each of the video, audio and subtitle data signals 

50 js zero in the data encoding apparatus. 

When the subtitle data for one page is read out of the display memory 22-2 (Fig. 1 3), the headers of the bit streams 
are separated therefrom by a parser 22-3 and the remaining data is forwarded to the inverse variable-length coder or 
run-length decoder 23, 24 during a vertical blanking period (V). Inverse VLC (Variable Length Coding) circuit 23 (Fig. 
2) subjects the subtitle data to variable length decoding. The variable length decoded subtitle data is composed of 

55 level data ("1" or "0") and run data as paired data. In the case where variable length decoding is not employed, the 
inverse VLC circuit may be bypassed and the subtitle data read from the code buffer will be directly output to inverse 
run-length circuit 24. Inverse run-length circuit 24 conducts run-length decoding by generating the level of data from 
the number of run data elements. Thus, VLC circuit 23 and run-length circuit 24 decompress the subtitle data which 
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had been stored as compressed data in code buffer 22. 

The decompressed subtitle data is then sent to a 3:4 filter 25. The 3:4 filter receives an xsqueeze signal from the 
system controller 1 4 indicating the aspect ratio of the corresponding television monitor. Where the signal indicates that 
the monitor has a 4:3 aspect ratio, the 3:4 filter applies 3:4 filtration processes to the subtitle data to match the size of 

5 the subtitles to the size of the video picture. In the preferred embodiment, the controller 35 reads 90 pixels worth of 
subtitle data from the code buffer 22 before the H sync pulse is generated. In the case where the television monitor 
already has a 16:9 aspect ratio, or the decompressed subtitle data represents fonts, the 3:4 filter is bypassed. 

A color look-up table 26 (CLUT) receives the subtitle data from the 3:4 filter 25 and the CLUT_data from the code 
buffer 22. The color look up table generates a suitable color from the CLUT_data for the subtitle data The color look 

io up table selects an address corresponding to the subtitle data for each pixel and forwards a mixing ratio K and color 
components Y (luminance), C R (color difference signal R-Y) and C R (color difference signal B-Y) to the mixer 34. The 
color components Y, C R and C B , when mixed by mixer 34, at the mixing ratio K create a pixel with the color indicated 
by the color look up table. 

Background video data is incorporated in the arrangement of the color look-up table. For example, address 0 of 
*s the look-up table includes key data K having the value of 00 h; which means that the subtitle data will not be seen and 
the background video data will manifest, as shown by regions T1 and T5 in Fig. 7c. Addresses 1h to 6h of the look-up 
table include values of the key data K which increase linearly (20, 40 CO hexadecimal); which means that the subtitle 
pixels according to these addresses are mixed with the background data as shown by the regions T2 and T4 in Fig 
7c. Finally, addresses 8h to Fh of the look-up table include values of key data K of EOh; which means that the compo- 
se nents Y, Cr and Cb are mixed without any background video data as shown by region T3 in Fig. 7c. The color look-up 
table data is generated from the system controller and is previously downloaded to the CLUT circuit before decoding. 
With the color look-up table, the filtered subtitle data is transformed into the appropriate color pixel for display on the 
television monitor. 

Fig. 8 shows one example of a color look-up table where the components Y, Cr, Cb and K are arranged according 

25 to the addresses 0...F (hexadecimal). As will be explained, color wiping is performed by changing the CLUT_data, 
thereby replacing part of the color look up table by the color wiping color look up table shown in Fig. 11. Normally, a 
particular subtitle frame is refreshed several times because frames are refreshed in a television signal several times 
a second. When the subtitles are refreshed, the same subtitle data will be employed. However, the color will be different 
due to the changed color look up table. Thus, the subtitles will appear to be color wiped as they are refreshed with 

30 each consecutive frame. 

A mixer 34 (Fig. 2) mixes the pixels from the color look-up table 26 with video data from video decoder 3 (Fig. 1 ). 
The resulting mixed data represents a video picture with superimposed subtitles and is ready to be output to a television 
monitor. Mixer 34 is controlled to position the subtitles within the video picture by referencing a imposition signal 
generated by system controller 14 from commands of an operator via controller 35. The imposition value designates 

35 the vertical position for display on the screen and may be varied (either by a user, the transmitter, or otherwise) allowing 
a user to place the subtitles anywhere along a vertical axis. 

The illustrative decoding apparatus of the present invention may be practiced with the parameters for the different 
signals shown in Fig. 6. However, the present invention is not limited to the parameters set forth in that figure and may 
be employed in different video systems. 

40 With the illustrative apparatus of the present invention, a user has control over the display of the subtitle through 

a mode display device 9 (Fig. 1). System controller 14, upon command from the user, sends a control signal to mixer 
34 (Fig. 2), turning the subtitles on or off. Since the present apparatus decodes subtitles in real time, the user does 
not experience any unpleasant delay when turning the subtitles on or off. In addition, the subtitles can be controlled, 
by the user or otherwise, to fade-in/fade-out at a variable rate. This is achieved by multiplying a fade coefficient to the 

45 pattern data representing the subtitles at a designated speed This function also allows an editor of the subtitles to 
present viewers with different sensations according to the broadcast of the audio/video picture For example, news 
information may be "flashed" rapidly to draw the attention of the viewer, whereas subtitles in a slow music video "softly" 
appear in order not to detract from the enjoyment of the music video. 

50 Subtitle Searching 

The illustrative method and apparatus of the present invention provides a viewer with the capability to search for 
a specific scene or sound quickly and efficiently. For example, the viewer may wish to quickly locate a specific dialogue 
in a video recording of an office meeting without watching lengthy periods of relative inactivity in the meeting. The 
55 present method and apparatus provides subtitles exclusively displayed during the trick playback mode and the viewer 
can quickly locate the desired dialogue by scanning such subtitles during the trick playback mode. Preferably, only 
those video frames which have subtitles displayed in the trick playback mode are played back, skipping over the scenes 
of relative inactivity to the important scenes. The subtitles for display in the trick playback mode may be narrative in 
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that a single page of subtitles corresponding to one video frame may be displayed over the course of a plurality of 
video frames to describe an entire scene or dialogue. In this manner, the viewer can easily locate any information in 
the video picture desired quickly and efficiently and need not wait through periods of relative inactivity. 

An example of displaying subtitles from bit streams of data decoded from the record medium is shown in Figs. 4A 

5 and B, wherein the bit streams include video bit streams (V_l, V_P; V_B), audio bit streams (A) and subtitle bit streams 
(Sp : Sw). Reproduction of the subtitle (Sp) for normal playback mode is shown in Fig. 4A, where it will be noticed that 
the subtitle (Sp) for the normal playback mode is divided into three bit streams which must be recombined to comprise 
one page of subtitles for a respective video frame. When the tape is moving in a forward direction in the normal playback 
mode, the bit streams of the subtitle (Sp) are reproduced by causing the pickup to jump to the individual bit streams 

10 of the subtitle as indicated by the arrows extending from left to right in Fig. 4A. Similarly, when the tape is moving in a 
reverse direction in the normal playback mode, the bit streams of the subtitle (Sp) are reproduced by causing the pickup 
to jump to the bit streams according to the arrows extending from right to left. After the bit streams for the subtitle (Sp) 
are reproduced, the decoder of Fig. 2 decodes the subtitle (Sp) for display with the corresponding video frame. 

According to illustrative apparatus and method of the present invention, subtitles (Sw) to be displayed in the trick 

15 playback mode, which may be different in textual content from the subtitle (Sp), are recorded adjacent to video frames 
(VJ). It will be appreciated that not all video frames (V_l) will be displayed in the trick playback mode and do not have 
an adjacent subtitle as shown in Fig 4B The subtitle decoder of the present embodiment during a forward trick play- 
back mode, will decode each subtitle (Sw) in the direction of the arrows from left to right in Fig. 4B and, similarly, decode 
in the direction of the arrows from right to left in reverse trick playback mode. Notably, the third video frame (VJ) is 

20 skipped because that video frame does not have a subtitle (Sw) since, for example, that video frame has little activity 
With the present apparatus and method, the subtitles displayed in trick playback mode may be of different textual 
content from the normal playback subtitles This means that the trick mode subtitles may include "special" information 
describing the scene or corresponding sounds. Indeed, the trick mode subtitles may be played back without the normal 
playback subtitles or the video picture, which provides the viewer with the ability to quickly find the desired information 

25 by scanning only the trick mode subtitles. Thus, the present apparatus and method provides quick and efficient location 
of information in a video picture by skipping to those video frames which have significant information. 

The present method and apparatus, during encoding, stores a starting sector address where the page header 
portion of the subtitle bit streams are located in areas reserved for addresses on the record medium for later retrieval 
during decoding. A first embodiment stores the addresses of the subtitles to be displayed in trick playback mode in a 

30 table of contents (TOC) (Figs. 3A, B) located at the beginning of the disc. The table of contents is not encoded and, 
therefore, is read directly by the system controller to drive the pickup to the addresses corresponding to the subtitles 
to be displayed in trick playback mode. 

The table of contents shown in Fig. 3A identifies frames by frame number (subcode frame #) and pointers (POINT) 
pointing to a track on the disk at which the corresponding frame is located, and time codes (PMIN, PSEC, PFRAME) 

35 corresponding to that frame. The table of contents identifies pages of stream data by indicating the frame, starting 
sector address, and ending sector address of recorded pages. From this table of contents, the system controller can 
locate the beginning of the subtitle pages and cause drive control 15 (Fig. 1 ) to jump the pickup to the sector indicated 
by the start_sector address in the table of contents. 

As shown in Fig. 3B the apparatus and method of the present invention may also store in the "trick playback 

40 duration" column, the display time data for the subtitles to be displayed. The controller (Fig. 2) determines the duration 
of display of the subtitle for various playback speeds using display time data by changing the multiplicative factor of 
multiplier 350 which factors the display time data in proportion to the playback speed. 

In a second embodiment of the invention, the sector addresses for the page header portions are collected in an 
area of the disk called a stream map. Similar to the previous embodiment the addresses here are directly employed 

45 by the system controller (Fig 1 ) without the need for decoding Unlike the table of contents, however, the stream map 
is not necessarily confined to the beginning of the disk, but may be located at any sector. The stream map is : thus, 
arranged as a packet with video, audio, subtitle, blanking, packet length of stream, identifier, and length of stream map 
information. The system controller references the stream map in a similar manner to the table of contents, thus causing 
the pickup to reproduce pages of subtitles by sending the reproduced pages of subtitle streams to the subtitle decoder. 

50 in a third embodiment, the sector addresses of the previous and following pages of subtitle streams are stored in 

each page header portion of a currently reproduced page. Since the sector addresses are in the page header portions, 
the sector addresses are sent to the subtitle decoder to be decoded, not the system controller. The word detector 20 
of the subtitle decoder (Fig. 2) detects the subtitle stream sector addresses (subtitle stream sector address) and 
forwards them to the system controller, via controller 35. As each subtitle is decoded in, for example, the forward 

55 playback mode, the system controller recalls the following sector address for the next page and causes the pickup to 
jump to the sector indicated by that following address to reproduce the next page of subtitles. Similarly, in the reverse 
playback mode, the system controller recalls the previous sector address for the previous page and causes the previous 
page of subtitles to be reproduced. Specifically, the word detector detects whether a stream includes sector addresses 
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according to the following operation: 



No. u£ bits 
1 



10 



15 



20 



user_data_t lag 

if (user_data_f lag = W 1 H )( 

length_of _user_daua ie 

nexc_subnicle_adciress_of f Bet 3 2 

ruaerved y 

previous_subt:ii_.le_address_of f set 2 4 

reserved 9 

] 



Mnemonic 
uimsbf 

bslbf 
bslbf 
bslbf 
bslbf 
bslbf 



25 



30 



35 



System controller 14 (Fig. 1 ) acts as a computer and executes the above operation, causing the word detector to 
determine if the user_data_flag is set to "1 11 and, if so, treats the next 1 6 bits as length_of_user_data; the next 32 bits 
as next_subtitle_address_offset; the next 8 bits as reserved; the next 24 bits as previous_subtitle_address_offset; and 
the last 8 bits as reserved. The word detector forwards this information to the system controller, via the controller 35, 
and continues detecting subtitle streams. The system controller receives the subtitle page sector addresses from this 
operation and controls the decoding apparatus as described. 

The record medium, thus, functions as a computer-readable memory which directs the system controller as a 
computer to playback the subtitles during a trick playback mode. The addresses as shown in, for example, Figs. 3A 
and B, act as pointers which direct the computer (i.e., -system controller 14) to skip to a location on the record medium 
where a subtitle to be displayed in the normal playback mode is recorded. During the trick mode playback, the pointer 
directs the computer to skip to a location on the record medium where a subtitle to be displayed during the trick playback 
mode is recorded. More specifically, the pointers cause the system controller to drive the drive controller 15 to move 
a pickup (Fig. 1 ) to jump to the indicated addresses of the record medium and read the subtitles. It will be appreciated 
that the computer-readable memory may be, for example, a video disc, a tape medium, or the like. 



Encoding Technique 

40 The encoding technique employed in the present illustrative apparatus and method will be described in more 

particular detail with reference to Figs. 7A, 7B, 7C and Fig. 8. As an example, the technique for encoding the letter "A" 
of Fig. 7A will be explained. The letter "A" is scanned along successive horizontal lines and the fill data of Fig 7B is 
generated for the letter "A" along each horizontal line. It will be noted that the level "EO" demarks the highest level for 
recreating a color pixel from the color look-up table shown in Fig. 6, whereas level "0" represents a lack of subtitle data. 

45 The key data (K) (or mixing ratio) determines the degree to which the fill data is mixed with background video 

Regions T1 and T5 of the key data correspond to areas in the video picture that are not superimposed with the fill data; 
therefore, these areas are designated as level 0 as indicated by address 0 in Fig. 8. Regions T2 and T4 are mixed 
areas where the subtitles are gradually mixed with the background video picture so that the subtitles blend into the 
background video picture and do not abruptly contrast therewith. Any of the fill data in this area is stored in addresses 

50 1 through 6 of the color look-up table. The main portion of the letter "A" is displayed within the T3 region where the 
background information is muted. The subtitle information in region T3 is stored as addresses 7 to F (hexadecimal). 
The color look-up table of Fig. 8 is arranged in varying degrees of the luminance component Y When a pixel in the 
region T3 is to be stored, for example, and the level of the luminance component Y for that particular pixel is 20 
(hexadecimal), the color information for that pixel is obtained from address 9 (Fig. 8). In this manner, the remaining 

55 pixels for the subtitle characters are encoded. 
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Encoding Apparatus 

The illustrative encoding apparatus of the present invention is depicted in Fig. 9. Audio and video information is 
received by a microphone 53 and video camera 51 , respectively, and forwarded to a multiplexer 58. The subtitle data 

5 are entered through either a character generator 55 or a flying spot scanner 56 and encoded by a subtitle encoding 
circuit 57. The encoded subtitle information is sent to multiplexer 58 and combined with the audio/video information 
for recording onto a record disc 91 or supplied to a channel for transmission, display recording or the like. 

Video camera 51 generates the video signal and supplies the same to a video encoding unit 52 which converts 
the video signal from analog to digital form. The digitized video signal is then compressed for video transmission and 

10 forwarded to a rate controller 52a, which controls the rate that the compressed video data is transferred to the multi- 
plexer in synchronism with the rate that the subtitles are sent to the multiplexer. In this manner, the compressed video 
data is combined with the subtitle data at the correct time. Similarly, audio information is obtained by microphone 53 
and encoded by an audio encoding unit 54 before being sent to the multiplexer. The audio encoding unit does not 
necessarily include a rate controller because the audio data may ultimately be recorded on a different track ortrans- 

15 mitted over a different channel from the video data. 

The subtitles are generated by either character generator 55 or flying spot scanner 56. The character generator 
includes a monitor and a keyboard which allows an operator to manually insert subtitles into a video picture The 
operator edits the subtitles by typing the subtitles through the keyboard Flying spot scanner 56, on the other hand, is 
used for the situation where subtitles are already provided in an external video picture or scanned in as text. The flying 

20 spot scanner scans the video picture and determines where the subtitles are positioned and generates corresponding 
subtitle data therefrom. The subtitles from the flying spot scanner are pre-processed by a processing circuit 63 to 
conform with subtitles generated by the character generator before further processing by the subtitle encoding circuit. 

The subtitle data from either character generator 55 or flying spot scanner 56 are, then, selected for compression. 
The character generator outputs blanking data, subtitle data and key data. The subtitle data and key data are forwarded 

25 to a switch 61 which is switched according to a predetermined timing to select either the subtitle or key data. The 
selected data from switch 61 is filtered by a filter 72 and supplied to another switch 62. Switch 62 switches between 
blanking data, the filtered data from the character generator, and the processed data from the flying spot scanner. 
When it is determined that no subtitles are present, the blanking data is chosen by switch 62. Where subtitles are 
present, switch 62 chooses between the character generator data or the flying spot scanner data, depending upon 

30 which device is being used to generate the subtitle data. 

The data selected by switch 62 is quantized by a quantization circuit 64, using a quantization level based on data 
fed back from a subtitle buffering verifier 68. The quantized data, which may be compressed, is supplied to a switch 
69 and (during normal operation) forwarded to a differential pulse code modulation (DPCM) circuit 65 for pulse code 
modulation. The modulated data is run-length encoded by a run-length coding circuit 66, variable-length encoded by 

35 a variable-length encoding circuit 67 and forwarded to the subtitle buffering verifier 68 for final processing before being 
sent to multiplexer 58. 

Subtitle buffering verifier 68 verifies that the buffer is sufficiently filled with data without overflowing. This is done 
by feeding a control signal (referred to in Fig. 9 as a filter signal) back to the quantization circuit. The control signal 
changes the quantization level of the quantization circuit thereby changing the amount of data encoded for a particular 

^o subtitle. By increasing the quantization level, the amount of data required for the subtitle data is reduced and the bit 
rate of data flowing to the subtitle buffering verifier is consequently reduced. When the subtitle buffering verifier deter- 
mines that there is an underflow of data, the control signal decreases the quantization level and the amount of data 
output from the quantization circuit increases, thereby filling the subtitle buffering verifier. 

The subtitle buffering verifier is also responsible for preparing the subtitle data for transmission (over television 

^5 airwaves, for example) The subtitle buffering verifier, to this end, inserts information necessary to decode the encoded 
subtitle data This information includes a normal/special play signal which indicates whether the subtitles are recorded 
in a normal or special (fast-forward/reverse) mode (referred to above as the trick mode). An upper limit value signal 
is inserted which indicates the upper limit for the memory size of the subtitle data for a frame. An EOP signal marks 
the end of page for the subtitle data frame and also is inserted. A time code signal is inserted which is used as the 

50 time stamp PTS in decoding. Subtitle encoding information is inserted and includes information used in encoding 
the subtitle data, such as the quantization factor. Positional information is inserted and is used as the position_data 
upon decoding. A static/dynamic signal is inserted which indicates whether the subtitle data is in static or dynamic 
mode. The subtitle buffering verifier also inserts the color look up table address for transmission to the decoder so that 
the colors of the display will match the colors employed in creating the subtitles. 

55 The subtitle buffering verifier is preferably a code buffer similar to the code buffer of the decoder (Fig. 2). To that 

end, it is useful to think of the operation of the subtitle buffering verifier to be in symmetry (i.e., performing the inverse 
functions of the code buffer) with the code buffer. For example, the color pixels of the subtitles are converted into digital 
representations; the resultant digital subtitles are encoded by the run length encoder and the variable length encoder; 
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header information is added; and the resultant subtitle information is stored in a buffer and forwarded to multiplexer 
58 for multiplexing with the audio and video data. 

Multiplexer 58 preferably employs time-sliced multiplexing; and also provides error correction processing (e.g., 
error correction coding) and modulation processing (e.g., EFM, eight-to-fourteen modulation). The multiplexed data is 
5 then transmitted (via television broadcasting, recording, or other means of transference) to the decoding apparatus for 
decoding and display. 

Encoding Subtitles 

10 The illustrative method and apparatus of the present invention permits a viewer to search for a specific scene or 

audible speech by providing subtitles to be exclusively displayed in the trick playback mode. The manner in which the 
encoder, already described with reference to Fig. 9, encodes the addresses will now be discussed. After multiplexer 
58 multiplexes the audio, video and subtitle data streams, the multiplexed data is sent to a sectorizing processor 1 00 
which arranges the data streams into fixed length sectors of packets. At this point, the data streams are ready for 

is airwave transmission. When the data streams are to be recorded on a disc, however, a table of contents (TOC) & 
stream map generator 101 determines the addresses of the pages of data streams to be recorded on the disk. 

According to the first embodiment, the TOC & stream map generator generates the table of contents shown in 
Figs 3A, B from the sectors generated by the sectorizing processor and the video/audio search information generated, 
for example, by a viewer In the second embodiment, the TOC & stream map generator generates the stream map 

20 from the sectors generated by the sectorizing processor. Unlike the previous embodiment, the TOC & stream map 
generator inserts the stream map as a packet onto the disk. In the first two embodiments, system controller 14 of the 
data reproducer (or receiver) reads the table of contents or the stream map directly and causes the decoding apparatus 
(Fig. 1 ) to decode the streams which relate to the data type being searched. In the third embodiment, the TOC & stream 
map generator inserts the previous and following page addresses into each of the page header portions of the pages. 

25 Unlike the first two embodiments, the system controller must cause the subtitle decoder to decode the page of subtitle 
streams and extract therefrom the sector addresses. As described, the TOC & stream map generator encodes each 
stream with 1 bit of a user_data_flag that indicates whether stream addresses are forthcoming in the stream; the next 
16 bits as length_of_user_data; the next 32 bits as next_subtitle_address_offset; the next 8 bits as reserved; the next 
24 bits as previous_subtitle_address_offset; and the last 8 bits as reserved. 

30 According to illustrative embodiments of the present invention, the video image, the audio track, and the subtitles 

are arranged in units of pages (i.e. frames) on the disc and the system controller accesses information by recalling 
from the disc the pages of streams. With this scheme, the system controller can cause the decoder to decode only 
those subtitles which correspond to videof rames of relatively significant information. Thus, a viewer can browse through 
the video picture by reading the subtitles and view only those portions which are of interest. 



35 



Colorwiping Encoding 



Colorwiping refers to a process by which an image, such as the subtitles, is gradually overlaid with another image. 
An exemplary application of colorwiping is highlighting, wherein a frame of subtitles is dynamically highlighted from 

40 left to right with the passage of time. I llustrative embodiments of the present invention perform colorwiping by changing 
the color look up tabic at different points of time during the subtitle display. For example, an initial subtitle frame is 
generated with the standard color look up table in Fig. 8. When colorwiping is performed, the color look up table is 
changed to the color wiping look up table of Fig. 1 1 . With the passage of each frame, the gradual change of the position 
at which the color look up table is changed from the colorwiping to the standard color look provides the sensation that 

45 the subtitles are changing color dynamically over time from left to right. 

An encoding operation for color wiping will now be discussed with reference to Figs 9 and 10 During the course 
of encoding subtitles, an operator may desire to color wipe the previously encoded subtitles. To that end, the operator 
is provided with a wipe lever 81 to control the colorwiping and a monitor 84 to view the color wiping in real time. The 
wipe lever is connected to an adapter 82 to adapt the analog voltages of the wipe lever to digital signals suitable for 

so digital manipulation. The digital output of the adapter is fed to both a switcher 83 and a wipe data sampler 70. The 
switcher switches the color look up table to values represented by the position of the wipe lever and generates color 
pixels of the subtitles for display on the monitor. Thus, the operator can visually inspect the colorwiping procedure while 
it occurs and adjust the speed or color of the wiping to satisfaction. 

The wipe data sampler and position sampler 70 determines from the adapter signals where in the video picture 

55 the color look up table is to be changed and outputs this information to encoding circuits 65, 66 and 67 (via switch 69) 
for encoding and transmission to multiplexer 58. Figs 1 0A and 1 0B depict a block diagram of the operation of the wipe 
data and position sampler. A comparator 301 compares a present pixel signal generated by the adapter with a previous 
pixel signal from the adapter. This is achieved by transmitting the present pixel value to input A of comparator 301 
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while supplying the previous pixel value latched in a register 300 to input B of comparator 301 . The comparator outputs 
a boolean "true" value to a counter 302 (which is reset at every horizontal or vertical sync pulse) when the present and 
previous pixels have the same value and, in response thereto, the counter increments a count value. That is, the 
comparator registers a true condition when the pixels up until that point are generated from the same color look up 
5 table. At the point where the color look up table changes, therefore, the present and previous pixels become unequal 
(i.e., their color changes) and the comparator generates a "false" boolean condition. The count value, thus, is equal 
to the number of matches between the present and previous values, which is the same as the position at which the 
color look up table changes. The count value is latched by a register 303 upon the following vertical sync pulse and 
transferred to the encoding circuits (via switch 69) for transmission. 

10 

Colorwiping Decoding 

Color wiping decoding will now be discussed with reference to Figs. 14A-C and 15. Fig. 14A shows the position 
here the color look up table is switched at point A from a color wiping look up table (Fig. 1 1 ) to the standard color look 

15 up table (Fig 8). Fig. 14B depicts a pattern of subtitle and colorwipe data arranged in discrete blocks of presentation 
time stamps (PTS(n) ... PTS(n+t)). The first presentation time stamp PTS(n) corresponds to normal subtitle data and 
the remaining presentation time stamps PTS(n+1 ... n+t) correspond to colorwiping data (WPA ... WPZ). Fig. 14C 
shows successive frames (n ... n+t) which correspond to the presentation time stamps To execute colorwiping, each 
successive colorwiping frame (WPA ... WPZ) sets the point where the color look up table is switched (point A) further 

20 along the displayed subtitle, thereby dynamically performing colorwiping as a function of time. 

An operational block diagram of the colorwiping decoding is depicted in Fig. 15. The vertical sync pulse triggers 
a register 205 to latch the current subtitle frame from a display buffer (Fig. 15 shows a colorwiping frame WP being 
latched). The colorwiping data latched by the register indicates the position of the color look up table switching. A pixel 
counter 208 decrements the value indicated by the colorwiping data at each horizontal sync pulse and outputs a boolean 

25 "true" flag to color look up table 26. While the flag is "true" the color look up table employs the colorwiping table (Fig. 
1 1 ) to decode the colors of the subtitle pixels. When the pixel counter reaches zero, the position of color look up table 
switching is reached and the pixel counter issues a boolean H false" flag to color look up table 26. At this time, the color 
look up table switches the colorwiping color look up table (Fig. 11) to the standard look up table (Fig. 8), and the 
remainder of the subtitle frame is displayed in standard color mode. Each successive colorwiping frame (WPA WPZ) 

30 moves the position of switching; thus ; each refreshed subtitle frame advances (or retreats) the colorwiping, thus per- 
forming dynamic colorwiping. 

The colorwiping color look up table in Fig. 11 incorporates two sets of colors (one set for addresses Oh to 7h and 
a second set for addresses 8h to Fh). Thus, the colorwiping color can be changed to a secondary color simply by 
changing the most significant bit (MSB) of the color look up table address. For example, the first set of colorwiping 

35 colors has a MSB of "0", while the second set has a MSB of "1 ". Changing the MSB of address 7h to a "1 " transforms 
the address to Fh and the colorwiping color changes. This may be done, for example, by setting the MSB equal to the 
flag of pixel counter 208. 

Employing the MSB to change between color sets has the advantage of reducing the number of bits required to 
be encoded. Since the MSB is known, only the three lower order bits need to be encoded where 4 bits are employed 
40 for every pixel. Where two bits are employed for every pixel, the subtitle data is coded only for the least significant bit. 
In a 4 bits per 2 pixel format, only the MSB is employed for color control and the remaining three bits can be reserved 
for pixel information. Thus, by using the MSB the number of bits encoded can be decreased and the overall processing 
time for encoding and decoding is optimized. 

45 Dynamic Subtitle Positioning 

The subtitles are repositioned dynamically, i.e., as a function of time, by employing a similar technique as described 
above with reference to colorwiping. As shown in Figs. 1 6 A-C and 1 7 the position data is measured along the horizontal 
axis (Fig. 16A) and is transferred to the subtitle decoder with the subtitle data during the appropriate frame (Fig. 16C) 
50 corresponding to a presentation time stamp (PTS(n) for example; Fig. 16B). 

The positioning operation will now be explained with reference to Fig. 17. The position data is a value representing 
the position of the subtitle frame along the horizontal axis and is read out from the display buffer and latched by register 
205 on each vertical sync pulse. Pixel counter 208 decrements the position data on each horizontal sync pulse and 
send a boolean flag to controller 35 (Fig. 2) to indicate that the position of the subtitle frame has not been reached. 
55 When the pixel counter reaches zero, the position of the subtitle frame has been reached and the boolean flag is 
toggled to indicate this to the controller. The controller, which has been delaying the reading operation ol code buffer 
22 (Fig. 2), then causes the code buffer to read out the subtitle data to run length decoder 24 (Fig. 2). The subtitle data 
is then decoded as described above and displayed with the corresponding video image. In this manner, the position 
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of the subtitle frame is changed with each frame; thus providing dynamic movement of the subtitle frame. 

It will be appreciated that the present invention is applicable to other applications, such as television or video 
graphics. It is, therefore, to be understood that, within the scope of the appended claims, the invention may be practiced 
otherwise than as specifically described herein. 

Claims 

1 . A method of storing subtitles composed of bit streams of subtitle data on a record medium having sectors arranged 
by address, said subtitles being stored with encoded video data constituting a video picture composed of a series 
of video frames, comprising the steps of: 



writing a first subtitle to be displayed with said video frames during a normal playback mode on the record 
medium at a first address; 

15 writing said first address on said record medium in an area reserved for addresses of the record medium; 

writing a second subtitle, different in textual content from said first subtitle, to be displayed during a trick play- 
back mode, on the record medium at a second address; and 

writing said second address on said record medium in said area reserved for addresses. 

20 2. The method of storing subtitles of claim 1 , wherein the step of writing a second subtitle further comprises writing 
the second subtitle as a page of bit streams of subtitle data beginning with a page header portion positioned at 
said second address on the record medium. 

3. The method of storing subtitles of claim 2, wherein the step of writing said second subtitle further comprises en- 
25. coding display time information within said page header portion indicating a display time for the second subtitle. 

4. The method of storing subtitles of claim 1 , further comprising the step of writing a table of contents on a contiguous 
area of sectors of said record medium preceding the encoded video data that stores said first and second addresses 
for assisting retrieval of said subtitles during playback. 

30 

5. The method of storing subtitles of claim 1 , further comprising the step of writing, on said record medium, a stream 
map as one of said bit streams of subtitle data that stores a record of said first and second addresses with the 
encoded video data lor assisting retrieval of said subtitles during playback. 

35 6. The method of storing subtitles of claim 1 , further comprising the step of writing said second address as one of 
said bit streams of subtitle data on said record medium. 

7. An apparatus for writing subtitles composed of bit streams of subtitle data on a record medium having sectors 
arranged by address, said subtitles being recorded with encoded video data constituting a video picture composed 

JO of a series of video frames, comprising: 

subtitle writing means for writing a first subtitle to be displayed with said video frames during a normal playback 
mode on the record medium at a first address, and for writing a second subtitle, different in textual content 
from said first subtitle, to be displayed during a trick playback mode on the record medium at a second address; 
45 and 

multiplexing means for multiplexing the first subtitle, the second subtitle and the encoded video data; 
wherein subtitle writing means writes the multiplexed data on said record medium, said first and second sub- 
titles being written at said first and second addresses, respectively, and writes the first and second addresses 
on said record medium in an area reserved for addresses on the record medium. 

50 

8. The apparatus of claim 7, wherein the subtitle writing means writes the second subtitle on said record medium as 
a page of bit streams of subtitle data beginning with a page header portion at said second address. 

9. The apparatus of claim 8, wherein the subtitle writing means writes display time information within said page header 
55 portion indicating a display time for the second subtitle during the normal playback mode. 

10. The apparatus of claim 7, wherein said means for writing further comprises means for writing a table of contents 
of the first and second addresses on a contiguous area of said record medium preceding the encoded video data 
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for assisting retrieval of said subtitles during playback. 

11. The apparatus of claim 7, wherein said means for writing further comprises stream mapping means for writing, as 
one of said bit streams of subtitle data, the first and second addresses on said record medium with the encoded 

5 video data, for assisting retrieval of said subtitles during playback. 

12. The apparatus of claim 7, wherein the means for writing writes the encoded second address in a bit stream of 
subtitle data on said record medium. 

10 1 3. A computer-readable memory for directing a computer to play back a subtitle to be displayed during a trick playback 
mode comprising: 

first pointing means for directing said computer during a normal playback mode to skip to an address pointing 
to a sector of said computer-readable memory where a first subtitle to be displayed with a respective video 
frame is stored and for causing said computer to read said first subtitle; and 

second pointing means for directing said computer during a trick playoack mode to skip to an address pointing 
to a sector of said computer-readable memory where a second subtitle, being different in textual content from 
said first subtitle and to be displayed with said video frames, is stored and for causing said computer to read 
said second subtitle. 

The computer-readable memory of claim 13, further comprising: 

a driven pickup movable to different positions for reading said first ana second subtitles from said computer 
readable memory in response to address signals produced by the first anr! second pointing means. 

25 15. The computer-readable memory of claim 14, wherein said computer-readable memory is a digital video disc. 

16. A method ol receiving subtitles composed of bit streams of subtitle data storeo on a record medium having sectors 
arranged by address, said subtitles being recorded with encoded video data constituting a video picture composed 
of a series of video frames, comprising the steps of: 

30 

reading out a subtitle address stored on said record medium pointing to a subtitle to be displayed during trick 
playback mode; 

reading out said subtitle stored on said record medium at said subtitle address when said subtitle address is 
read out; and 

35 decoding said subtitle for display with said respective video frame during said trick playback mode. 

17. The method of claim 16, further comprising the steps of: 

reading out display time data stored with the subtitles on said record medium indicating a duration in which 
40 subtitles are to be displayed in a video frame; and 

varying said duration for different playback speeds of the trick playback mode of the video picture such that 
said subtitle is displayed for a time determined by a particular playback speed. 

1 8. The method of claim 1 7, wherein the step of varying divides the display time data by twice the playback speed for 
45 the normal playback mode to speed up said duration as a function of the playback speed in trick playback mode. 

19. The method of claim 16, further comprising the step of reading out the subtitle to be displayed in trick playback 
mode stored on said record medium as a page of bit streams of subtitle data beginning with a page header portion 
positioned at said subtitle address on the record medium. 

50 

20. The method of claim 16, further comprising the step of reading a table of contents stored on said record medium 
in a contiguous area of sectors preceding the encoded video data that stores addresses of the subtitles to be 
displayed in trick playback mode for assisting retrieval of said subtitles. 

55 21. The method of claim 16, further comprising the step of reading, from said record medium, a stream map as one 
of said bit streams that stores the addresses of the subtitle to be displayed in the trick playback mode for assisting 
retrieval of said subtitles during decoding. 
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22. The method of claim 1 6, further comprising the steps of: 



reading said subtitle address stored on said record medium from one of said bit streams of subtitle data; and 
decoding said subtitle address stored on said record medium as video encoded data for assisting retrieval of 
$ said subtitles. 



23. An apparatus for recovering subtitles composed of bit streams of subtitle data from a record medium having sectors 
arranged by address, said subtitles being recorded with encoded video data constituting a video picture composed 
of a series of video frames, comprising: 

10 

means for reading, from said record medium, a subtitle address pointing to a subtitle to be displayed exclusively 
with a respective video frame during a trick playback mode and for reading said subtitle at said subtitle address; 
and 

means lor decoding said subtitle for display with said respective video frame during said trick playback mode. 

15 . 

24. The apparatus of claim 23, further comprising: 



means for reading out display time data stored with the subtitles on said record medium indicating a duration 
in which the subtitle is to be displayed in a video frame; and 
20 means for varying said duration for different playback speeds of the video picture such that said subtitle is 

displayed for a time determined by a particular playback speed. 

25. The apparatus of claim 24, wherein said means for varying is a multiplier which multiplies said display time data 
by a factor for varying said duration. 

25 

26. The apparatus of claim 25, wherein said factor is the inverse of twice the particular playback speed for speeding 
up the duration as a function of the playback speed. 

27. The apparatus of claim 23, further comprising means for reading the subtitle for display in the trick playback mode 
30 stored on said record medium as a page of bit streams of subtitle data beginning with a page header portion 

positioned at said subtitle address on the record medium. 

28. The apparatus of claim 23, wherein said reading means further reads a table of contents stored on said record 
medium in a contiguous area of sectors preceding the encoded video data which stores addresses of the subtitles 

35 to assist retrieval of said subtitles. 



29. The apparatus of claim 23, wherein said reading means further reads, from a stream map stored on the record 
medium, the addresses of the subtitles on the record medium for assisting retrieval of said subtitles. 

30. The apparatus of claim 23, wherein said reading means further reads said subtitle address as one of said bit 
streams of subtitle data from said record medium; and 

wherein, the decoding means further decodes said subtitle address. 

31. The apparatus of claim 30, further comprising a controller for driving a pick up to a sector of said subtitle address 
and for retrieving said subtitle from said record medium positioned at said subtitle address 
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FIG. 3A 
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FIG. 6 
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FIG. 8 
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FIG. 11 
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FIG. 18A 



FIG. 18B 
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